General Gittins index processes in discrete time.

General branching processes in discrete time as random trees

The simple Galton–Watson process describes populations where individuals live one season and are then replaced by a random number of children. It can also be viewed as a way of generating random trees, each vertex being an individual of the family tree. This viewpoint has led to new insights and a revival of classical theory. We show how a similar reinterpretation can shed new light on the more...

Discrete Time Stochastic Processes

4 Martingales 35 4.1 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 4.2 Doob Decomposition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 4.3 Optional Sampling Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 4.4 Inequalities and Convergence . . . . . . . . . . . ...

Switching Costs and the Gittins Index

Discrete Time Markov Processes

What follows is a quick survey of the main ingredients in the theory of discrete-time Markov processes. It is a birds' view, rather than the deenitive \state of the art." To maximize accessibility, the nomenclature of mathematical probability is avoided, although rigor is not sacriiced. To compensate, examples (and counterexamples) abound and the bibliography is annotated. Relevance to control ...

A Generalized Gittins Index for a Class of Multiarmed Bandits with General Resource Requirements

We generalise classical multi-armed and restless bandits to allow for the distribution of a (fixed amount of a) divisible resource among the constituent bandits at each decision point. Bandit activation consumes amounts of the available resource which may vary by bandit and state. Any collection of bandits may be activated at any decision epoch provided they do not consume more resource than is...